Autonomous trailer hitching using neural network

ABSTRACT

One aspect of the disclosure provides a method of maneuvering a vehicle in reverse for attachment to a trailer. The method includes: determining, at a computing device in communication with the neural network, a selected trailer in proximity to the vehicle; detecting at least one user input gesture performed by a user and captured in at least one image from at least one camera on the vehicle; selected a maneuver command for the vehicle based on the detected gesture; and executing the maneuver to move the vehicle from the initial position toward a final position adjacent the trailer.

TECHNICAL FIELD

This disclosure relates to an automotive vehicle configured to identifyone or more trailers positioned behind the automotive vehicle and driveto one of the one or more trailers.

BACKGROUND

Trailers are usually unpowered vehicles that are pulled by a powered towvehicle. A trailer may be a utility trailer, a popup camper, a traveltrailer, livestock trailer, flatbed trailer, enclosed car hauler, andboat trailer, among others. The tow vehicle may be a car, a crossover, atruck, a van, a sports-utility-vehicle (SUV), a recreational vehicle(RV), or any other vehicle configured to attach to the trailer and pullthe trailer. The trailer may be attached to a powered vehicle using atrailer hitch. A receiver hitch mounts on the tow vehicle and connectsto the trailer hitch to form a connection. The trailer hitch may be aball and socket, a fifth wheel and gooseneck, or a trailer jack. Otherattachment mechanisms may also be used. In addition to the mechanicalconnection between the trailer and the powered vehicle, in some example,the trailer is electrically connected to the tow vehicle. As such, theelectrical connection allows the trailer to take the feed from thepowered vehicle's rear light circuit, allowing the trailer to havetaillights, turn signals, and brake lights that are in sync with thepowered vehicle's lights.

Some of the challenges that face tow vehicle drivers are connecting thetow vehicle to the trailer, because more than one person is needed. Forexample, one person drives the vehicle, e.g., the driver, and anotherone or more people are needed to view the tow vehicle and the trailerand provide the driver with direction regarding the path the tow vehiclehas to take to align with the hitch. If the people providing directionsto the driver are not accustomed to hitching a tow vehicle to a trailer,then they may have difficulty providing efficient instructions fordirecting the path of the tow vehicle.

Recent advancements in sensor technology have led to improved safetysystems for vehicles. Arrangements and methods for detecting andavoiding collisions are becoming available. Such driver assistancesystems use sensors located on the vehicle to detect an ongoingcollision. In some examples, the system may warn the driver of one ormore driving situations to prevent or minimize collisions. Additionally,sensors and cameras may also be used to alert a driver of possibleobstacles when the vehicle is traveling in a forward direction.Therefore, it is desirable to provide a system that includes sensors toovercome the challenges faced by drivers of tow vehicles.

SUMMARY

One aspect of the disclosure provides a method of maneuvering a vehiclein reverse for attachment to a trailer. The method includes:determining, at a computing device in communication with the neuralnetwork, a selected trailer in proximity to the vehicle; detecting atleast one user input gesture performed by a user and captured in atleast one image from at least one camera on the vehicle; selected amaneuver command for the vehicle based on the detected gesture; andexecuting the maneuver to move the vehicle from the initial positiontoward a final position adjacent the trailer.

Implementations of the disclosure may include one or more of thefollowing optional features. In some implementations, determining, avehicle path from the initial position to the final position adjacentthe trailer, the vehicle path comprising maneuvers configured to movethe vehicle along the vehicle path from the initial position to thefinal position; following, at a drive system in communication with thecomputing device, the vehicle path from the initial position based upona user input gesture to begin path following; stopping or halting, atthe drive system, the vehicle when at least one of: a stop or halt inputgesture from the user is detected, or the vehicle is at an intermediateposition before reaching the final position, the intermediate positionbeing closer to the final position than the initial position; modifying,at the drive system, one or more vehicle suspensions associated with thevehicle to align a vehicle hitch with a trailer hitch based upon one of:the determined vehicle path and at least one a user input gesture tomodify the drive system or vehicle suspensions; and following, at thedrive system, the vehicle path from the intermediate position to thefinal position based upon a user input gesture to begin path following.

In some implementations, the maneuvers include, steering, braking, andspeeding. The method may further include: continuously detecting, by theneural network, one or more objects within the vehicle path as thevehicle is moving along the vehicle path. The method may also includewhen detecting an object, altering the vehicle path at the computingdevice. In some examples, detecting by a neural network of the vehicle agesture performed by the user in one or more images; and detecting, by aneural network of the vehicle, one or more trailers within one or moreimages and receiving, at a user interface in communication with theneural network, an indication of a selected trailer from the one or moredetected trailers. Detecting one or more trailers may include:capturing, at one or more imaging devices in communication with theneural network, one or more images, at least one of the one or moreimaging devices positioned on a back side of the trailer facing arearward direction; and determining, at the neural network, at least oneof: a user gesture and the one or more trailers within the at least oneimage. The details of one or more implementations of the disclosure areset forth in the accompanying drawings and the description below. Otheraspects, features, and advantages will be apparent from the descriptionand drawings, and from the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic view of an exemplary tow vehicle having a userinterface displaying an indication of trailers behind the tow vehicle.

FIGS. 2A-2B are schematic views of an exemplary tow vehicle.

FIGS. 3A-3D are perspective views of an exemplary vehicle hitchconnecting to a trailer hitch.

FIGS. 4A-4D are schematic views of a first set of exemplary usergestures for control input for the driver assistance system.

FIGS. 5A-5E are schematic views of a second set exemplary user gesturesfor control input for the driver assistance system.

FIG. 6 is a flow diagram of an exemplary arrangement of operations foroperating a tow vehicle in reverse for attachment to a trailer.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

A tow vehicle, such as, but not limited to a car, a crossover, a truck,a van, a sports-utility-vehicle (SUV), and a recreational vehicle (RV)may be configured to tow a trailer. The tow vehicle connects to thetrailer by way of a trailer hitch. It is desirable to have a tow vehiclethat is capable to autonomously maneuvering towards a trailer andattaching to the trailer, thus eliminating the need for a driver todrive the tow vehicle in a rearward direction while another one or morepeople provide the driver with directions regarding the path that thetow vehicle has to take to align with the trailer and ultimately a hitchof the trailer. As such, a tow vehicle with an autonomous rearwarddriving and hitching feature provides a driver with a safer and fasterexperience when hitching the tow vehicle to the trailer.

Referring to FIGS. 1-2B, in some implementations, a driver of a towvehicle 100 wants to tow a trailer 200. The tow vehicle 100 may beconfigured to autonomously drive towards the trailer 200. The towvehicle 100 may include a drive system 110 that maneuvers the towvehicle 100 across a road surface based on drive commands having x, y,and z components, for example. As shown, the drive system 110 includes afront right wheel 112, 112 a, a front left wheel 112, 112 b, a rearright wheel 112, 112 c, and a rear left wheel 112, 112 d. The drivesystem 110 may include other wheel configurations as well. The drivesystem 110 may also include a brake system 120 that includes brakesassociated with each wheel 112, 112 a-d, and an acceleration system 130that is configured to adjust a speed and direction of the tow vehicle100. In addition, the drive system 110 may include a suspension system132 that includes tires associates with each wheel 112, 112 a-d, tireair, springs, shock absorbers, and linkages that connect the tow vehicle100 to its wheels 112, 112 a-d and allows relative motion between thetow vehicle 100 and the wheels 112, 112 a-d. The suspension system 132improves the road handling of the tow vehicle 100 and provides a betterride quality by isolating road noise, bumps, and vibrations. Inaddition, the suspension system 132 is configured to adjust a height ofthe tow vehicle 100 allowing the tow vehicle hitch 160 to align with thetrailer hitch 210, which allows for autonomous connection between thetow vehicle 100 and the trailer 200.

The tow vehicle 100 may move across the road surface by variouscombinations of movements relative to three mutually perpendicular axesdefined by the tow vehicle 100: a transverse axis X, a fore-aft axis Y,and a central vertical axis Z. The transverse axis x, extends between aright side R and a left side of the tow vehicle 100. A forward drivedirection along the fore-aft axis Y is designated as F, also referred toas a forward motion. In addition, an aft or rearward drive directionalong the fore-aft direction Y is designated as R, also referred to asrearward motion. When the suspension system 132 adjusts the suspensionof the tow vehicle 100, the tow vehicle 100 may tilt about the X axisand or Y axis, or move along the central vertical axis Z.

The tow vehicle 100 may include a user interface 140, such as, a display142. The user interface 140 receives one or more user commands from thedriver via one or more input mechanisms 143, 145, 410, 410 a-d or atouch screen display 142 and/or displays one or more notifications tothe driver. The user interface 140 is in communication with a vehiclecontroller 300, which is in turn in communication with a sensor system400, in particular a camera(s) 410, 410 a-d. In some examples, the userinterface 140 displays an image of an environment of the tow vehicle 100leading to one or more commands being received by the user interface 140(from the driver) that initiate execution of one or more behaviors. Thevehicle controller 300 includes a computing device (or processor) 302(e.g., central processing unit having one or more computing processors)in communication with non-transitory memory 304 (e.g., a hard disk,flash memory, random-access memory) capable of storing instructionsexecutable on the computing processor(s)).

The vehicle controller 300 executes a driver assistance system 310,which in turn includes a path following sub-system 320. A gesturerecognition system 560 interprets gestures (movement) made by a userinto commands for the diver assistance system 310. A path planningsystem 550 then determines a path 552 based upon the input gestures. Thepath following sub-system 320 receives the path 552 from the pathplanning system 550 and executes behaviors 322-330 that send commands301 to the drive system 110, leading to the tow vehicle 100 autonomouslydriving about the planned path 552 in a rearward direction R.

The path following sub-system 320 includes, a braking behavior 322, aspeed behavior 324, a steering behavior 326, a hitch connect behavior328, and a suspension adjustment behavior 330. Each behavior 322-330cause the tow vehicle 100 to take an action, such as driving backward,turning at a specific angle, breaking, speeding, slowing down, amongothers. The vehicle controller 300 may maneuver the tow vehicle 100 inany direction across the road surface by controlling the drive system110, more specifically by issuing commands 301 to the drive system 110.For example, the vehicle controller 300 may maneuver the tow vehicle 100from an initial position (as shown in FIG. 3A) to a final position (asshown in FIG. 3C). In the final position, a hitch ball 162 of the towvehicle 100 aligns with a hitch coupler 212 of the trailer 200connecting the tow vehicle 100 and the selected trailer 200.

The tow vehicle 100 may include a sensor system 400 to provide reliableand robust autonomous driving. The sensor system 400 may includedifferent types of sensors that may be used separately or with oneanother to create a perception of the tow vehicle's environment that isused for the tow vehicle 100 to autonomously drive and make intelligentdecisions based on objects and obstacles detected by the sensor system400. The sensors may include, but not limited to, one or more imagingdevices (such as cameras) 410, and sensors 420 such as, but not limitedto, radar, sonar, LIDAR (Light Detection and Ranging, which can entailoptical remote sensing that measures properties of scattered light tofind range and/or other information of a distant target), LADAR (LaserDetection and Ranging), etc. In addition, the camera(s) 410 and thesensor(s) 420 may be used to alert the driver of possible obstacles whenthe tow vehicle 100 is traveling in the forward direction F or in therearward direction R, by way of audible alerts and/or visual alerts viathe user interface 140. Therefore, the sensor system 400 is especiallyuseful for increasing safety in tow vehicles 100 which operate undersemi-autonomous or autonomous conditions. While the camera(s) 410 arepart of the sensing system 400 they may also act as part of the userinterface 140, by way of the gesture recognition system which performimage analysis on the recorded images to interpret the gestures made bythe user.

In some implementations, the tow vehicle 100 includes a rear camera 410,410 a that is mounted to provide a view of a rear driving path for thetow vehicle 100. Additionally, in some examples, the tow vehicle 100includes a front camera 410, 410 b to provide a view of a front drivingpath for the tow vehicle 100, a right camera 410, 410 c positioned onthe right side of the tow vehicle 100, and a left camera 410, 410 dpositioned on the left side of the tow vehicle 100. The left and rightcameras 410, 410 c, 410 d provide additional side views of the towvehicle 100. In this case, the tow vehicle 100 may detect object andobstacles positioned on either side of the tow vehicle 100, in additionto the objects and obstacle detected along the front and rear drivingpaths. The camera(s) 410, 410 a-d may be a monocular camera, binocularcamera, or another type of sensing device capable of providing a view ofthe rear travelling path of the tow vehicle 100.

In some implementations, the tow vehicle 100 includes one or more NeuralNetworks (NN) 500, for example, Deep Neural Networks (DNN) to improvethe autonomous driving of the tow vehicle 100. DNNs 500 arecomputational approaches used in computer science, among otherdisciplines, and are based on a large collection of neural unites,loosely imitating the way a biological brain solves problems with largeclusters of biological neurons connected by axons. DNNs 500 areself-learning and trained, rather than programed, and excel in areaswhere the solution feature detection is difficult to express in atraditional computer program. In other words, DNNs 500 are a set ofalgorithms that are designed to recognize patterns. DNNs 500 interpretsensor system data 402 (e.g., from the sensor system 400) through amachine perception, labeling or clustering raw input. The recognizedpatters are numerical, vectors, into which all-real-world data, such asimages, text, sound, or time series is translates. The DNN 500 includesmultiple layers of nonlinear processing units 502 in communication withDNN non-transitory memory 504. The DNN non-transitory memory 504 storesinstructions that when executed on the nonlinear processing units 502cause the DNN 500 to provide an output 506, 508. Each nonlinearprocessing unit 502 is configured to transform an input or signal (e.g.,sensor system data 402) using parameters that are learned throughtraining. A series of transformations from input (e.g., sensor systemdata 402 in the form of user gestures) to outputs 506, 508 occurs at themultiple layers of the nonlinear processing units 502. Therefore, theDNN 500 is capable of determining the location based on images 412 orsensor data 422 eliminating the need to have a DGPS or a GPS.

The DNN 500 receives sensor system data 402 (including images 412 and/orsensor data 422) and based on the received data 402 provides an imageoutput 506 to the user interface 140 and/or a data output 508 to thevehicle controller 300. In some examples, the DNN 500 receives image(s)412 of a rear view of the tow vehicle 100 from the camera 410 incommunication with the DNN 500. The DNN 500 analyzes the image 412 andidentifies one or more gestures by the user in the received image 412.The DNN 500 may also receive sensor data 420 from the sensors 420 incommunication with the DNN 500, and analyze the received sensor data420. Based on the analyzed images 412 (or the analyzed images 412 andthe sensor data 422), the DNN 500 identifies the a perceived commandfrom the user, for example by way movement relative to a coordinatesystem.

In some examples, the user interface also may include a touch screendisplay 142. In other examples, the user interface 140 is not atouchscreen and the driver may use an input device, such as, but notlimited to, a camera, a rotary knob or a mouse In one example embodimentthe user interface 140 may be a combination of a wireless device toprovide prompts, and signals to the user and may further includereceiving images from the cameras 410, 410 a-d, wherein the user maymake gestures whose meaning can be interpreted through image analysis,as explained in further detail below.

When the driver performs a command gesture the gesture recognitionsystem 560 interprets the meaning of the gesture, e.g. stop, move right,etc, and the path planning system 550 plans a path 552 between the towvehicle 100 and the trailer 200 based on the gestures (determined by theDNN 500 from the received sensor system data 402).

The tow vehicle 100, therefore, is autonomously following the gesturesand backing up towards the selected trailer 200.

In some examples, the gesture recognition system 560 a and path planningsystem 550 are part of the vehicle controller 300 as shown in FIG. 2A;while in other examples, gesture recognition system 560 b is part of theDNN 500 as shown in FIG. 2B. Referring to FIG. 2A, when the userperforms a gesture viewed by the camera(s) 410, 410 a-d, the gesturerecognition system 560, 560 a located in controller 300 determines thecommand associated with that particular gesture. In this case, the pathplanning system 550 a receives the information and plans the next stepin the path 552 between the tow vehicle 100 and the selected trailer200. The path planning system 550 a may use several methods to determinethe path 552.

Referring back to FIG. 2B, in some implementations the DNN 500 includesthe gesture recognition system 560, 560 b. As such, the gesturerecognition system 560, 560 b may determine the command associated withthat gesture based on learned behaviors. For example, the DNN 500determines the between variations in how the gesture is performed, whichmay occur among different users, slight movement, exaggerated movement,arms/hand held higher or lower, etc.

Referring back to FIGS. 2A and 2B, once the path planning system 550plans a path 552, the path following sub-system 320 is configured toexecute behaviors the cause the drive system 110 to autonomously followthe planned path 552. Therefore, the path following sub-system 320includes one or more behaviors 322-330 that once executed allow for theautonomous driving of the tow vehicle 100 along the planned path 552.The behaviors 322-330 may include, but are not limited to a brakingbehavior 322, a speed behavior 324, a steering behavior 326, a hitchconnect behavior 328, and a suspension adjustment behavior 330.

The braking behavior 322 may be executed to either stop the tow vehicle100 or to slow down the tow vehicle based on the planned path 552. Thebraking behavior 322 sends a signal or command 301 to the drive system110, e.g., the brake system 120, to either stop the tow vehicle 100 orreduce the speed of the tow vehicle 100.

The speed behavior 324 may be executed to change the speed of the towvehicle 100 by either accelerating or decelerating based on the plannedpath 552. The speed behavior 324 sends a signal or command 301 to thebrake system 120 for decelerating or the acceleration system 130 foraccelerating.

The steering behavior 326 may be executed to change the direction of thetow vehicle 100 based on the planned path. As such, the steeringbehavior 326 sends the acceleration system 130 a signal or command 301indicative of an angle of steering causing the drive system 110 tochange direction.

Referring to FIGS. 5A-5D, in some examples, the tow vehicle 100 isautonomously maneuvered along the planned path 552 until the tow vehicle100 reaches an intermediate position P_(M) being an intermediatedistance D_(M) from the selected trailer 200, as shown in FIG. 3A. Inthe intermediate position P_(M), the tow vehicle hitch 160 is in anorientation aligned generally parallel with the selected trailer 200 andthe tow vehicle hitch 160 is substantially aligned with the hitch 210 oftrailer hitch 210. In other words, the vehicle fore-aft Y defines aplane that extends along the vehicle vertical axis Z and along thetrailer fore-aft T along a trailer vertical axis. In some examples, theintermediate distance D_(M) is about 1 meter.

When the tow vehicle 100 is in the intermediate position P_(M) a hitchconnect behavior 328 may be executed to connect the vehicle hitch 160with the trailer hitch 210. The user determines a relative height H_(R)between a top portion of the tow vehicle hitch ball 162 and a bottomportion of the trailer hitch coupler 212. To connect the tow vehicle 100and the selected trailer 200, the trailer hitch coupler 212 releasablyreceives the tow vehicle hitch ball 162. Therefore, to connect the towvehicle hitch ball 162 to the trailer hitch coupler 212, the relativeheight H_(R) has to equal zero allowing the tow vehicle hitch ball 162to move under and be inserted in the trailer hitch coupler 212.Therefore, when the user notices relative height H_(R) that is greaterthan zero between the tow vehicle hitch ball 162 and the trailer hitchcoupler 212 from the user performs a gesture which can be interpreted bythe DNN and/or controller 300 to adjust the suspension system (e.g.lowering arm toward the ground), the gesture recognition system 560, 560a, 560 b sends a command to the suspension adjustment behavior 330 toexecute and issue a command 301 to the suspension system 132 to adjustthe height of the tow vehicle 100 reducing the relative height H_(R).When the user perceives the relative height H_(R) that is equal to zero,then the user performs another gestures which is interpreted by the DNN500/controller 300 which issues a command 301 to the drive system 110 tomaneuver along the remainder of the path 552, i.e., from theintermediate position P_(M) to a final position P_(F) (FIG. 3D),connecting the tow vehicle 100 to the selected trailer 200.

Referring to FIGS. 1, 2A, 2B and FIGS. 4A-4D, a control system 560, 560a, 560 b and method for providing user input is described. In thisembodiment the user can control the Auto-Hitch Driver Assistance System310 by making various gestures. As previously discussed, the gesturerecognition system 560, 560 a, 560 b located in the DNN 500 orcontroller 300 receives images 412 from camera 410, 410 a-d and/orsensor data 422 from sensors 420 and based on the received data 402provides an image output 506 to the user interface 140 and/or a dataoutput 508 to the vehicle controller 300. In this embodiment userinterface 140 may be and/or include communication with a wireless deviceand/or audible prompts which the user can receive outside of the towvehicle 100.

In this example, the gesture recognition system 560, 560 a, 560 blocated in the DNN 500 or controller 300 receives image(s) 412 of a rearview and possibly side view of the tow vehicle 100 from the camera 410,410 a-d in communication with the gesture recognition system 560, 560 a,560 b located in the DNN 500 or controller 300. The gesture recognitionsystem 560, 560 a, 560 b located in the DNN 500 or controller 300analyzes the image 412 using the processing units 502, 502 a-n andidentifies one or more gestures in the images, which are performed bythe driver in view of the camera 410, 410 a-d. Gesture recognition andspecific meaning associated with each gesture may be carried out bytraining the DNN 500 in the manner of DNN 500 learning previouslydiscussed.

Based on the analyzed images 412 (or the analyzed images 412 and thesensor data 422), the gesture recognition system 560, 560 a, 560 blocated in the DNN 500 or controller 300 identifies the gestures andassociates the gesture with a given command, e.g. stop, pause, move inreverse, lower suspension, raise suspension, move forward, move to theright direction, change move to the left direction, repeat path planningand final alignment path following, etc. Gestures may be selected to be“intuitive” of those typically used in providing direction to drivers ofbacking trailers. Arm gestures, as illustrated in FIGS. 4A-D, or similarhand gestures as illustrated in FIGS. 5A-E may be used. The gesturerecognition system 560, 560 a, 560 b located in the DNN 500 orcontroller 300 sends the command to the path planning system 550, 550 bwhich interprets the behavior and sends to the path following sub-system320 to follow the maneuver as directed by the command associated withthe detected gesture.

Further, analysis of image 412 is also used to determine a location ofthe user performing the control gestures relative to the tow vehicle100, and to the planned path once determined, and the processing units502, 502 a-n or the path planning system 550, 550 b confirm the user isin a safe location prior to providing the signal/command 301 to thedrive system 110. If the user is not in a safe location the user may beprompted through image, text and/or audible command prior to release ofthe tow vehicle 100 for movement. The analysis of image 412 and safetycheck of the user relative to the planned path of the tow vehicle 100will them be repeated prior to release of the tow vehicle 100.Alternatively or in addition, the user may provide input the userinterface 140 that they are now in a safe location and the safety checkimage analysis should be repeated. Additionally to stay within view ofthe camera 410, 410 a-d and to stay in a safe location the user may needto move as the tow vehicle 100 moves, indicated by arrow.

In one embodiment, the driver assistance system 310 is designed to workreal-time in which the tow vehicle 100 moves as the user inputs thedirections. Therefore, the image analysis for gesture recognition by thegesture recognition system 560, 560 a, 560 b located in the DNN 500 orcontroller 300, path planning and user position safety check arecontinuously repeated during movement of the tow vehicle 100 to thefinal trailer location.

FIGS. 4A-D and 5A-E illustrate example gestures that the user may employwhile using the vehicle 100 and trailer 200. The gestures shown are onlymeant to be exemplary in nature, one skilled in the art would be able todetermine gestures that are commonly or intuitively known in directingvehicles. Further, selected gestures which are not commonly known may beselected and taught to the gesture recognition system 560, 560 a, 560 blocated in the DNN 500 or controller 300 for any command, e.g. a commandspecific to backing a vehicle to a trailer which does not already have acommon gesture.

FIGS. 4A-D include gestures that may involve movement by the user inmaking the gesture, e.g swing arms in one direction or another, circlinga hand to indicate repeat, pushing a hand or hands forward to indicatestop or pause. FIGS. 5A-E illustrate gestures that do not involvementand the stationary hand and/or arm positions give meaning to therequested input.

Referring to FIG. 4A, the user moving their left hand toward the left,may indicated a desire to change the movement in that direction, suchthat the vehicle 100 moves more in that direction. In FIG. 4B, the usermoving their right hand toward the right which may indicate a desire tochange the movement in that direction, such that the vehicle 10 movesmore in that direction. The user moving one or both hands toward theirbody, shown in FIG. 4C, may indicate a desire to continue in the currentdirection of movement. The user moving or holding one or both hands awayfrom their body may indicate a desire to stop, FIG. 4D. Other arm andhand gestures may also or alternatively be employed.

Referring to FIG. 7A, the user crossing the forearms, may indicated adesire to stop motion of the vehicle 100. In FIG. 7B, the user holds outa flat hand with fingers extended which may indicate a desire to pausemotion of the vehicle 100. The control system 310 can therefore,distinguish between a desire to stop, e.g. place the car in park and/orstop all autonomous motion and the desire for the vehicle 100 to merelypause, e.g. hold the brake while waiting further instructions. Themaking a first with one hand, shown in FIG. 7C, may indicate a desire tocontinue motion in the current direction of movement or to continuemotion after a pause. Further a thumbs up from the user, FIG. 7D, mayindicate a desire to raise the vehicle suspension and a thumb down formthe user, FIG. 7E, may indicate a desire to lower the vehiclesuspension. Other arm and hand gestures may also or alternatively beemployed.

FIG. 6 illustrates an example arrangement of operations for a method 600of autonomously maneuvering a tow vehicle 100 (as shown in FIGS. 1-5)towards a selected trailer 200, 200. At block 602, the method 600includes receiving an indication that a driver wants to autonomouslyhitch the tow vehicle 100 to a trailer 200 and initiating a hitch assistmode. The indication may be by way of a selection on the user interface140 of the tow vehicle 100, putting the tow vehicle in reverse (withoutreversing), or any other indication. At block 604, a camera 410, 410 a-ddetects a gesture performed by a user in view of the camera. At decisionblock 606, the gesture recognition system 560, 560 a, 560 b performsimage analysis determine a vehicle command associated with that gesture.In some examples, the gesture recognition system 560, 560 a, 560 b maybe part of the controller 300 or part of the DNN 500. The requestedcommand is send from the gesture recognition system 560, 560 a, 560 b toa path planning system 550, 550 a, at block 608, the method 600 includesplanning a path 552 based on the command. At block 610, the method 600includes executing the path following sub-system 320. At decision block612, the user determines if the tow vehicle 100 is within apredetermined distance from the selected trailer 200, i.e., theintermediate position P_(M). When the tow vehicle 100 reaches theintermediate position P_(M), the user at decision block 612 determines arelative height H_(R) between a top portion of the hitch ball 162 of thetow vehicle 100 and a bottom portion of the hitch coupler 212 of theselected trailer 200 and determines if the hitch coupler 212 canreleasably receive the hitch ball 162 based on the relative heightH_(R). In other words, the user determines if the relative height H_(R)equals to zero. If the relative height H_(R) is not equal to zero, thenat block 616, the user adjusts the suspension of the tow vehicle 100through the appropriate gesture which is interpreted by the gesturecontrol system (and ultimately sent to the suspension adjust behavior330. The user repeats until they determines the relative height H_(R)and checks if the relative height H_(R) equals zero at block 614. Oncerelative height H_(R) is equal to zero, then the method 600 at block 618continues maneuvering about the path 552 from the intermediate positionP_(M) to a final position P_(F) connecting the hitch ball 162 of the towvehicle 100 with the hitch coupler 212 of the selected trailer 200.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium” and“computer-readable medium” refer to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

Implementations of the subject matter and the functional operationsdescribed in this specification can be implemented in digital electroniccircuitry, or in computer software, firmware, or hardware, including thestructures disclosed in this specification and their structuralequivalents, or in combinations of one or more of them. Moreover,subject matter described in this specification can be implemented as oneor more computer program products, i.e., one or more modules of computerprogram instructions encoded on a computer readable medium for executionby, or to control the operation of, data processing apparatus. Thecomputer readable medium can be a machine-readable storage device, amachine-readable storage substrate, a memory device, a composition ofmatter effecting a machine-readable propagated signal, or a combinationof one or more of them. The terms “data processing apparatus”,“computing device” and “computing processor” encompass all apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, or multiple processors or computers.The apparatus can include, in addition to hardware, code that creates anexecution environment for the computer program in question, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them. A propagated signal is an artificially generated signal, e.g.,a machine-generated electrical, optical, or electromagnetic signal thatis generated to encode information for transmission to suitable receiverapparatus.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multi-tasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made without departingfrom the spirit and scope of the disclosure. Accordingly, otherimplementations are within the scope of the following claims.

What is claimed is:
 1. A method of maneuvering a vehicle in reverse forattachment to a trailer, the method comprising: receiving an indicationof an autonomous hitch mode is started; detecting at least one firstgesture performed by a user and captured in at least one image from atleast one camera on the vehicle; determining a first command associatedwith the at least one first gesture using a gesture recognition system;planning a path based on the first command with a path planning systemand maneuvers to follow the path; executing the maneuvers to move thevehicle along the path from an initial position toward an intermediateposition being an intermediate distance along the path from a finalposition; at the intermediate position, receiving a second gestureperformed by the user, the second gesture including a suspensionadjustment gesture; determining a second command associated with thesuspension adjustment gesture using the gesture recognition system;executing a suspension adjustment behavior causing a suspension systemof the vehicle to adjust; receiving a third gesture performed by theuser; determining a third command associated with the third gestureusing the gesture recognition system; and executing the maneuvers tomove the vehicle along the path from the intermediate position to thefinal position adjacent the trailer.
 2. The method of claim 1, whereinthe intermediate position is closer to the final position than theinitial position.
 3. The method of claim 1, further comprising, at thefinal position, connecting a vehicle hitch with a trailer hitch.
 4. Themethod of claim 1, wherein the maneuvers include steering, braking, andspeeding.
 5. The method of claim 1, wherein the gesture recognitionsystem is located in a neural network device to learn a plurality ofgestures and associate each of the learned plurality of gestures with acommand for the vehicle.
 6. The method of claim 1, wherein the first andthird gestures include at least one of: a stop gesture, a pause gesture,a move in reverse gesture, a lower suspension gesture, a raisesuspension gesture, a move forward gesture, a move to the right gesture,a move to the left gesture, repeat path planning gesture, or finalalignment path following gesture.
 7. A method of maneuvering a vehiclein reverse for attachment to a trailer, the method comprising:capturing, from at least one camera supported by a rear portion of thevehicle, images of a user; detecting a first gesture performed by theuser; determining, using a gesture recognition system, a first commandassociated with the first gesture and causing the vehicle to perform anaction; executing vehicle maneuvers causing the vehicle to move along apath from an initial position towards an intermediate position being atan intermediate distance along the path from a final position, thevehicle maneuvers based on the first command; at the intermediateposition, receiving a second gesture performed by the user, the secondgesture including a suspension adjustment gesture; determining, using agesture recognition system, a second command associated with thesuspension adjustment gesture; and executing a suspension adjustmentbehavior causing a suspension system of the vehicle to adjust.
 8. Themethod of claim 7, further comprising: receiving a third gestureperformed by the user; determining, using a gesture recognition system,a third command associated with the third gesture; and executing vehiclemaneuvers to move the vehicle along the path from the intermediateposition to the final position adjacent the trailer.
 9. The method ofclaim 8, further comprising, at the final position, connecting a vehiclehitch with a trailer hitch.
 10. The method of claim 7, wherein theintermediate position is closer to the final position than the initialposition.
 11. The method of claim 7, wherein the maneuvers includesteering, braking, and speeding.
 12. The method of claim 7, wherein thegesture recognition system is located in a neural network device tolearn a plurality of gestures and associate each of the learnedplurality of gestures with a command for the vehicle.
 13. The method ofclaim 7, wherein the first gesture include at least one of: a stopgesture, a pause gesture, a move in reverse gesture, a lower suspensiongesture, a raise suspension gesture, a move forward gesture, a move tothe right gesture, a move to the left gesture, repeat path planninggesture, or final alignment path following gesture.